FIG. 3 is a block diagram showing the arrangement of a conventional single-CCD digital camera, and especially shows a signal processor in detail.
A signal from a CCD image sensing element 501 is adjusted in white gain by a WB (White Balance) circuit 502, and sent to a luminance notch circuit 510. The luminance notch circuit 510 reduces the gain of color signal components near the Nyquist frequency in the vertical direction by using a VLPF (Vertical Low-Pass Filter), and also reduces the gain in the horizontal direction by using an HLPF (Horizontal Low-Pass Filter). These filters are called luminance notch filters. Then, an HBPF (Horizontal BandPass Filter) circuit 511 and VBPF (Vertical BandPass Filter) circuit 514 shift up frequency components slightly lower than the Nyquist frequency decreased by the notch filters.
PP (aperture Peak) gain circuits 512 and 515 adjust the amplitude in both the horizontal and vertical directions, and BC (Base Clip) circuits 513 and 516 cut small-amplitude components and remove noise. An adder 517 adds the horizontal and vertical components, an APC (APerture Control) main gain circuit 518 applies a main gain to the resultant signal, and an adder 519 adds a baseband signal to the signal. A γ conversion circuit 520 γ-converts the signal, and a luminance correction (YCOMP) circuit 521 corrects the luminance signal level by color.
Assume that an output from an image sensing element having filters of a checkered Bayer layout as shown in FIG. 4 is processed. In particular, primary color filters achieve good color separation. Therefore, as shown in FIG. 5A, the notch filter method cannot absorb gain differences between different color filters by using only LPFs at the edge of an image having opposite hues, e.g., red and blue in left and right halves. The edge staircases or becomes jaggy, which degrades the image quality of a playback image. This will be explained with reference to FIG. 5B.
FIG. 5B is a view for explaining an output level from each pixel of the image sensing element. In FIG. 5B, a pixel outputting a relatively large value is blank, and a pixel having an output of almost 0 is hatched for descriptive convenience. Signal level differences between different color filters are large at an edge between opposite hues, and appear as jaggies. Further, the jaggies are enhanced by edge enhancement which is performed to increase the resolution (MTF: Modulation Transfer Function) which has been decreased by LPFs.
To prevent jaggies, the following adaptive interpolation luminance signal generation method has been proposed. The correlation between signals of upper, lower, right, and left pixels of the pixel to be interpolated is detected to determine based on the correlation whether this pixel corresponds to a vertical or horizontal stripe. Interpolation is performed using signals of upper and lower pixels for a vertical stripe, and using signals of right and left pixels for a horizontal stripe, thereby preventing jaggies of a luminance signal.
The adaptive interpolation luminance signal generation method will be described with reference to the flow chart of FIG. 8.
A green signal is interpolated first. For example, in the image shown in FIG. 6A, to interpolate pixels P1 to P9 (each parenthesized character represents a chroma signal obtained from the pixel, and corresponds to the color of the filters), a green signal at the pixel P5 (P5(G)) is interpolated as follows.
1. The absolute values (HDiff and VDiff) of the differences between upper and lower pixels of the pixel to be interpolated and between right and left pixels of the pixel to be interpolated are calculated by equation (1) (step S101):HDiff=|P4(G)−P6(G)|, VDiff=|P2(G)−P8(G)|  (1)
2. The interpolation method is changed based on the calculated absolute values (step S102).
If VDiff<HDiff, the green signal P5(G) is interpolated using signals of adjoining pixels in the vertical direction by equation (2) (step S103):P5(G)=(P2(G)+P8(G))/2  (2)
If VDiff>HDiff, the green signal P5(G) is interpolated using signals of adjoining pixels in the horizontal direction by equation (3) (step S104):P5(G)=(P4(G)+P6(G))/2  (3)
Green signals are interpolated in this way for pixels which output signals other than green signals. Thereafter, red and blue signals are interpolated. Red signals are calculated byP2(R)=((P1(R)−P1(G))+(P3(R)−P3(G)))/2+P2(G)P4(R)=((P1(R)−P1(G))+(P7(R)−P7(G)))/2+P4(G)P5(R)=((P1(R)−P1(G))+(P3(R)−P3(G))+(P7(R)−P7(G))+(P9(R)−P9(G)))/4+P5(G)  (4)Blue signals can also be obtained by the same calculation. Accordingly, signals of three, R, G, and B colors can be attained for each pixel. Further, a luminance signal Y is calculated byY=0.3×R+0.59×G+0.11×B  (5)
On the other hand, a color interpolation circuit 503 interpolates a chroma signal so as to give all color pixel values to all pixels (step S105). A color conversion MTX (MaTriX) circuit 504 converts the R, G, and B signals to the chroma signal. A CSUP (Chroma SUPpress) circuit 505 suppresses the chroma gain in low and high luminance regions, and a CLPF (Chroma Low-Pass Filter) circuit 506 limits the band of the chroma signals. A γ conversion circuit 507 converts the band-limited chroma signal into R, G, and B signals and at the same time γ-converts the R, G, and B signals. The γ-converted R, G, and B signals are converted into a color difference signal again. A C gain knee (Chroma gain knee) circuit 508 adjusts the chroma gain of the color difference signal, and an LCMTX (Linear Clip MaTriX) circuit 509 corrects the hue. The resultant signal is output together with the luminance signal.
Matrix operation by the color conversion MTX circuit 504 satisfies conditions that no false color is generated in the vertical direction when an image of an achromatic object is sensed.
However, in matrix operation performed by the color conversion MTX circuit 504, a false color is readily generated at a high frequency around the Nyquist frequency.